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O any tabSe{s) related to sequence listing (specify): 
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1 Reference \& made to 



US200at)14228S A1 
US2003/0095197A1 
US2003/0044070 A1 



^ Jn^tt^n^f '^""^-^ .""'^^ ^'"''^ foHowing two groups of dafms fail to share 
correspondmg special technfcal features defining a common inventive concept 

SlTon ?fT """^"^''i'^ ^^^"^'^^^ ^«*^^*'"9 *-«^-^y^s in an acquired imaqe 

based on a ftrst acquired .mage which has been previousjy con-ected If ana Ss o^^ 
satd first image determined this to be beneficial ^«freciea jt analysis of 

Group 2: Claims 25^7: (if oiarified) relate to analysing a first acquired imaae f«r 
adaptiveiy modifying a filter chain to detect red eye defects in anlcSrS 

^' ^ SncSpt^''* '^^^^ ^"""P^ ^'^'"^^ ^^''^ ^ ^<'"^^on inventive 

^'^ Saivsis fmm STrSSlf ^'1^*^^ "^^"9 information extracted by image 

analysis from a first image to jnfluence the overall process of red ©ye detection ir^^n 
image: thss rs known from both D1 and D2. w.rt D1 cf i a Fin p- if «r^« 

sL'ntS?^K'^°"^ ^ influencing 55 Ind iSlga cf t gg 1 ref 

Signs 62,64 influencing the processing chain 70. ^' 

^"^ I^if f ^1?'"^ ^""^ concepts are not common to both sets of ciaims In 

particular there is no mention of a filter chain in qroup 1 , nor is there anv SLinn 

white in group 1 the characteristics are used to correct first Image data wh ch is th«n 
used to fnfiuence the processing of an Image, in group 2 thriSmSsttes ar^ ttLr 
used to modify a f ilter chain - see i.a. also pg. 14%nf3 m,Sy^^^^^ 

^f^^^^Tr^^T^if"^^''' - adapts theVed eyi flliffSafn!" and 
t I S? ">od'fy the filter chain ... as opposed to modification of the im2n^ 

b^nt fn,^"^°" T**"* ^"^ "•'^ remaining concepts differ thenD can 
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Re ttem V 



Reasoned statement with regard to novefty. Inventive step or industriar aooflcahiiifw 
citations and explanations supporting such statement applicability, 

3.1 A contradicfcn between step f) of claim 1 and the overaij disctosure (cf i a 

ttr/cif 'V''^' ^^^^ always ap^led 

to the corrected .mage whereas red eye correction may be applied to another 

Ihat^l^tS?'^ '''^ ""^'^ '^'S- t(b). pg 16 lines 5?9). Th s en^ms 

25 S?e?lfinX3^^^ '^f ff/^ T ^" '^^9^ t^^^^S^^ '^^'^ ^ known 
and bette! defined quafrty: cf. pg. 2 { 8-14 and notably pg. 3 lines 8-1 1 and do 
1 1 hne 30-pg. 12 ilne 3, pg. 12 line 17-pg. 13 fine 12 pq 1 5 HnL 20 ?rnn 

^'y- 2(b),4(a). Thiffn turn i?essi;.' JirTo 

problem as discussed on pp. 1-3 of the description, in contrast to this Dresant 
claim 1 hints at red-eye detection being performed on a sec^ndlmag;*^ Sre^^ 
tS^nrnl?/^^^^^ image with the latter however also somehow be^fg 'u2dT 
Snd Inth! iSr?'""'. ^"^"^^""^^S disclosure for such a scenario lould be 
found m the app .cat.on documents which teach applying the red-eye detection 
ShtLt^r^'^^K 'T^^:. '^"^^^^ ^'^^ convincing masons for proceedSg 
Snir . ^xK^^ ^"'"."'^ S^^-'®^^^ knowledge nor in tHe applicaion 

documents The application does mention that, in the special case of 
Hnf images detection may be applied to uncorrected images (pg. 19 

Hne 6 and pg^23 l,ne 19 ). but note that even in this case the system rs neSer 
applymg a red-eye detector to a different image while simultanLusiy also us no 
data from a (non-existent) Wected" first Image in the process of deteSon 

^'^ SthTfiJf V-"" ""^^ ^V^^ "corrective processes" are restricted to corrections 
'^^^f'^'^^^l^^^^^^^^'^oBment {cf. pg, 2 fine 12). They couid presentjy be 
read on complete red-eye corrections perfomed on the first Imaqe in 
SnSSf Vr^ description. This demonstrates that the disctosure is not 
sufficremiy enabling over the full scope presently attributable to the cSaimed 
subject matter and that the term "corrective process" applied in claim 1^ 
ambiguous in the present context. 

For the above reasons the subject matter of present claim 1 could presently 

flZ^T'T'^f S-^- '^"^'^^^ '^^g^ sequences comprising at 

east a first and a subsequent second image once one interprets the express on 
corrective processing" as red eye correction in the first f rarne of the sequence 
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f^SSiu^.'^^K "^^^^-^ w?^'^ '^"^ ^y^^ "^^^ correction would 

certamjy be benef ,aal} and if one interprets "detecting red ©ye defects In saM 
tlll?l''.rf^^ u^^ng said corrected first imago" as referring to the Ssuli 
t ackeg perfoj-med m mage sequences, Ib. searching for a region In the next 
frame .n the vicinity of important regions of the previous frame ffence thr 

ft ctear and complete disciosure can be found and the 

present^ claimed subject matter doss not find enabjing support over fuH 
width Moreover, the subject matter of claim 1 therefore also^aifdTntSJ^^^^ 
novelty {or at least an inventive step) w.r.t. red-eye detection in imfJe 
sequences combined with eye tracking. ^mQciion m image 



Ts t^^^ ^ - ~d subject 

" "''nTrfi'.;^?® "r^mty^ng .. the ... image J' in claims 1 and 25 e.g. as 

.^perfotming irnage analysfs operations on the .„ image pixeis.. " (basis cf 
«.a. pg. 2 Une 10) as othemise the dairned subject matter could also be " 
mts-mad on analysing the image data onfy w.r.t its aSociald melt^^^^ 
as anticipated by D2 ref, signs 66,68 of Fig 2 ^"^'^^^^^ meia data 

" ^jy*^'^^® u ^ formulation of step c) in claim 1 which is vague and 
I V'^T/r to which referenclscale 

beneffaai" shou d be gauged. Moreover, the claim wording could aiso be 
mis~read on unrelated subject matter, e.g. on detemiining if one is abfe to 
process the image at ail - e.g. if memory constraints aiio5 one to dj so 
annIC nn. ' ^^P'""^"^' evin on detemiining i^oni shoulS ^ 

apply only one as opposed to mom cortBctive pn^cesses {"if one or more 
), The overall disclosure indicates that one rather deteirnines accordiL 
Za^a^^^'V ohaijoteristfcs if and which correctivrpTess or ^ 
processes could possibly improve image quality. 

^ i!pl ^J^^Tr '^'^ description seemingly Indfcates on 

pp. 24-25 that the ''3rd princrple embodiment" (color convetBion) mav Dossiblv 

SrS"^o r'^-*^^ P""«'P^f embodiments. No cSms at^ 

«nS i!? *? S:'^'^'^^' embodiment per se (Note that such dZs would in 

"«"-""*ta'V stJfoJect matter and the third prindpll 
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3.5 The presence of two Independem methodi claims 1 and 25 would ateo Jead to an 
unclear and tnconciss set of cialniis when analysed as a whole because claims 

1 and 25 specify partialiy different features as well as apparently corresponding 
features in different terms, thereby rendering the actual matter for which 

protection is sought unclear. 

3.6 it is not stated that the methods are automated and computer implemented. 
Thus they could refer to mere mental schemes in the form of user instructions 
for interactive manual red eye removal 

3.7 In ciaim 25 there is no link between steps c) and d) implied because It is not 
mentioned that the adapting in step d) depends on the output of step c), 

3.8 In ciaim 26 the numbering of steps overlaps with the numbering of steps of 
claim 25 and the numbering of claim 26 also does not make clear that any 
corrective processes are applied before step b) of claim 25. 



Group 1: clafms 1-24 



4 tn light of the description, the subject matter of ciaim 1 must be interpreted as i,a. 
covering the following interpretation: 

1 . first create a re-sampied image of a main input image, e.g. subsampiod due to 
memoty constraints - cf, i.a. Fig. 1(b) and also pg. 10 lines 18-27, pg. 1 1 lines 
27-30, but note also that in view of Fig 1 (b) this step is actually even optional 
and one may always wori< on the main image, and 

2. then apply to the resulting Image any conventional image enhancement if 
needed, e.g. tone scale conversion to compensate for underexposure, cf. also 
pg. 13 lines 17-22 of the application documents, 

3. then apply to the enhanced image any conventional red eye detection scheme, 

4.1 Hence, the subject matter of claim 1 <in so far as it is in line with the overaii 

disclosure) lacks novelty w.r.t. D1 as can be seen from a comparison of Fig 1 (b) of 
the application + Fig. 2 of D1 (see also corresponding description of D1): 



Fig, 2 of D1 Fig 1 (b) of the Application 

ref, sign 40 corresponds to 112,170-3 

58,60,63 130 
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135-1,135-2 



Remarks: 



1. As for steps c) and d) of c?aim 1: as mentioned this may amount to det&rmmma 

17-22) befote performing red-eye detection. Cteariy such kind of nrocessSia 
implied by ref signs 58,60,63,50 of Rg. 2 of D1 . processing ts 

^' !nr£rf ^^''^SOing, the fact that D1 additionally also teaches a sepamte 
furthei fme scan processing chain 46 totalfy irreievant for this objeSon 

odtained in 51 to aid aSso in the red ©ye correction step m 55 wouid be trivia! in 
order to save space, time and to ensure harmonised rLuitfin lo and iT^^m • 
hen^ even the not yet expHcitfy claimed embodiment on pg. 16 Hnos s j o the 
present application documents wouid be obvious In fight ofDI 

4.2 The subject matter of cfaim 1 (in so far as it Is in line with the overali disrto^.im\ 
wouid additionally also fack an inventive step vv.r.t. D2 as c J, be Sen fSmT^^ 
companson of Rg 1 (b) of the appiication 4- Rg, 2 of 

Fig. 2 of D2 Rg 1 (b) of tfie Application 

ref . sign 62 corresponds to 1l2,i70-3 

64 ^30 

136-1,135-2 
80 90 

One might even argue that the present claim formulation Is antlciDated also bv no 
because depending on the obtained parameter values ?or e Sb^^^^ p 
D2 (e g. If the amount of ad|ustments happens to be zero) no cormctk^n S^^^^^^ 
and this should be sufficient to anticipate aiso steps cTand d^ SrcuSn?cte m 1 In 
any case, it would be a totally trivial design modification Sf D2 F g Jto ap^^^^^^^^ SBA 
t^lT-^T'l ^^r^^ ^ete^^t'vely if needed, especially since pamgraph K 
a^aady indicates the possibility to do so. Altematlveiy, it would be ISytS to 
combine D2 with the ideas of ref. sign 48 of Ro 2 of D1 and «T,«mltI^^^^ L 
image anaiysis steps influence no lo%er oIlyfhe^^BA Ihls STg'l of^D2 but'""' 
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rather also the decisions in the mage defect predfotion process 68 of Ra 2 D2 in 
from the image pfxel data Itseif instead of meta-data. ^ l«agea 

4.3 Moreover, the Interpretation of claim 1 cited above amounts anvwav to a nan. 
.nventive standard Juxtaposition of usually applied image p^oSq operStions 
(resampling, enhancing if needed, red-eye detection). ^"'^^^^'"S operattons 

5 As for the dependent claims: 

claims 2-4,6-24: known from D1 ,D2 ibid, 
claim 5: triviai parallelization 

Group 2: claims 25-47 

6 The subject matter of independent claims 25. 46,47 iacks novefty. 

6.1 The expression "adapting said filter chain" in steps c)+d) of claim 25 can sfmoiv stand 

^'^'^ Iv^f I""**^ the claimed subject matter lacks novelty w.r.t, image feature 

extracton and the subsequent use of these features for taking decisions in anv 
algonthm cons.sf.ng of various red-eye detection filtering subSeS Th?s ii 
common .n red-eye detection In general, for an example cf. cS Ra 1 n D3 
image features determine which processing route is eventually taken in a red- 
eml.tyer'' ^ ^^'"^ "red-eye subfirters" ev3y Sgit 

^'^ hISI*^®^"'?'"^ more sophisticated control, D1 teaches influencing the red-eye 

^" f^iShscan image ("second image") in accSnce 

w th m^age features extracted from the prescan Image ("first ima|e"), cl fn pSSr 
mJfrnLT [0054-0060] and note that section 63 rLik^s dSom 40 

fr^M^fi^nl^""^ ^''"^r? f '^"^-^^^ ^''^^^■^"Q 46 including seSSns 

S^K P?°t^^'"9 ^^^^ from 42. The claimed subject matter thus lacks noveftv 

image of D1 . Hence, even disregarding the above objections w.r.t lack of c aWtv and 
anabi,ng disclosure, and nevertheless assuming that one werrdeaSj 
.mages the subject matter of independent claims 25.46.47 would 5ilKn^^^^ 

6.3 Moreover the subject matter of claims 25,46,47 would also lack an inventive step 
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SS.lv mfs£^*.^ ?«"^r ''rfW ''^ « design modlflomion Sf D2 Rg a ,o 

7 As for the dependent claims: 

claims 26-28,30-45: known from D1/D2, se© above 
claim 29; trivial pamlleli'zation. 
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CLAIMS: 

L A method for red-eye detection in an acquired digital image comprising the steps of: 

a) acquiring a first image; 

b) analysing the first acquired image to provide a plur aiit>' of ohamcteristtcs of an image ; 
5 c) determining if one or more corrective precedes can be beneficiaHy afjpRcd to said first 

acquired image accoTding to said characteristics; 

d) applying any such determined corrective processes to ^id first acquired inside; 

e) acquiring a second diflbrent image; and 

f) detecting red-^sye delects in said second acquwed image using said connected fim acquired 
10 Image. 

2. A melhod according to claim I wherein said detecting step comprises applying a chain of 
one or mote red-eye filters to said first acquired image; md further comprising ttie stcj^ of, prior 
to said detecting step: 

15 g) detennining if said red-eye filter chain can be ad^ted in acconlance with said plurality of 
(^aracteristlcs; and 
h) adapting said redeye filter {^ainaccoidingty. 

3. A method according to claim 2 in which said step of adapting comprises providing an 
20 altered set of parameters for one or mixe filters of said ftiter chain. 

4. A method according to cl^m 2 in which said step of adapting comprises re-ordering a 
sequence in which said filters are applied to said first acquired image. 

25 5, A method according to claim 2 in which more than one of said filters are applied 

simultaneously, 

6. A roefeod according to claim 2 in which said step of ad^ting comprises determtmng 
which filters sm to be applied in said chain. 
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7. AmethodaccoKllng toolain, I fi^^her comprising the step of con^ti«g a thini acquit 
image based on any defects detected in said detecting step. 



A method according to claim 7 wherein said fmt second and Mrd 



images ao-e the same 



9. A method according ,o claim 7 « fc«, „^ of ^ ^ 

sub-sampled copy of said thiid image. 

10 

15 of the second image: 

n, A method according to claim I ferthercomprisingOiestepof: 
settings of a device used to acquire the second image. 

20 

loreoff ' -*^«-«^-'^«»«*y--S«t^ comprises dete^^^^^^^ 

a degree of blur; 
adepeeofdtt 
^5 color balance; 



a gamma comection which might be applied; 
texture characterlsticss; 

noise characteristics; and 
characteristics of regions; 
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in tfie Itrst acquired image. 



14. A method accoiding to claim 1 wherein said analysing step comprises recognising one or 
more faces or types of face in the ftfst acquired image. 



15. 

of: 



A method according to claim I wherein said corrective processes eomprise one w more 

(i) contmst normalization and image sharp^ing; 

(ii) image color adjusdnent and tone scaling; 
(Hi) exposure adjustment and digitaJ fill flash; 

(tv) brightness adjustment tvith cofor Space matching; 

(v) image auto-gamma determination with image enhancement; 

(v) image enhancement; and 

(vi) face based image eanhanconuant . 



J6. A method accordtng'to claim 2 wherein said t«d-«ye filter chmn comprises: 

(i) a pixel locator and segmentor; 

(ii) a sh^e analyser; 

(iii) a falsing analyser; and 

(iv) a pixel modilRer. 

I ?. A method according to claim 16 wherein said pixel locator and segmentor includes a 



25 IS. A method as ciaimed in claim I comprising the step of: responsive to determining that 
corrective processing is necessary but cannot be faeneficiaily applied to said first acquired image 
according to said characteristics, disabling said step of detecting red-esye defects and providing an 
indication of such to a user. 



3. 
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19. A method as claimed in claim 7 wfa^in said step of ^^iymg a corrective process 

ineludes interacting with a user to determine the corrections to be made to said third image. 

20. A method according to ctaim I wheretn the step of applying a corrective process includes 
performing a color space transformation. 

21 . A method as claimed In claim 1 in which one or more of steps a) to e> are performed in a 
digital cam^. 

22. A method as claimed in claim 1 ccmiprising determhilng a sequmce in which more tlian 
10 one corrective pnicess can be bene:6ci£d!y appHed to said tirst acquired image according to said 

characteristics. 

23. A computer-readable storage medium containing a set of instnictions which when 
executed on a digital image processing device perform the steps of claim 1 . 

15 

24. A digital image processing device airanged to perform the steps of claim 1 . 

25. A m^od for red-eye detection In an acqtiired digital image comprising the steps of; 
a) acquiring a first image; 

20 b) analysing the fiiDst acquired image to provide a plurality of characteristics of an tm^^^ 

c) detemiining if a chain of one or more red-eye filters can be adapted in accordance with 
said characteristics; 

d) adapting said red-eye filter chain; 

e) acquiring a second image; and 

25 0 a^piiytng said filter chain to detect red-eye defects in said second acquired image, 

26. A method according to claim 25 further comprising the steps of: 

f) determining if one or more cortective processes em be beneftciaily applied to said fiist 
acquired image according to said characteristics; and 
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g) «PPJy^«g any such con-ective processes to said first acquired image. 

27. A method according to claim 35 in which said step of adapting comprises providing an 
altered set of parameters for one or more filters of said filler chain. 

28. Aniethodaccordingtoclaim25inwhichsaidst^ofadapti„gcomprisesie^Mem^ 
sequence m which said filters are applied to said f 



29. Amethodaccordingtocla,m25inwhiohmot*.thanoneofsaidmte«are^plied 
10 simultaneously. 

30. A method acco«ltag » claim 25 in which «*l a»p rf«Mng campri«, deto^intog 

Which filters are to be applied in said chain. 

15 31, Amethodacoc^dingtoclaimasforthercomprisingthestepofcon^^^^^ 
tmage based on any defects tistected in said detecting step. 

32 Amethodacco«iingtoclaim31whereina£!eastoneofsaidfi,Bta«dsecondimagesi^ 
sub-sampled copy of said tbiid image. 

33. A method according to claim 25 wherem said first acqui^d image is a sub-sampled copy 
ofaa acquired image. 

25 Of the second image. 

35. A method according to claim 25 further comprising the step of: 

prior to analysing the first acquired image, processing said second image based on the 
setHngs of a device used to acquire the second image. 

30 



20 
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36. A method according to claim 25 wherein said analysing step comprises <ieten»i«ing one 
or mom of: 

a degree of blur; 
a ^gt^ of dust contamination; 
color balance; 
vt^ite balance; 

a gamma correction which might be applied; 
texture characferisdcs; 
noise characteristics; and 
characteristics of regions^ 
in the first acquired image. 

37. A method according to claim 25 wherein said analysing step comprises recognising one 
or more &ees or tj^es of face in the flr$t acquired image. 

3S. A mediod according to claim 26 wherein said coirective processes comprise one or more 
of: 

(i) contrast normatization and image sharpening; 

(ii) image color adjustment and tone scaling; 
20 (ill) exposure adjustment and digital fill flash; 

(iv) brightness adjustment with color space matching; 

(v) image auto-gamma detennination with image esn*taneemrait.; 
(v) Image enhancement; and 

(vt) face based image enhancement . 

25 

39. A method according to ciaim 25 wherein said red-eye filter chain comprises: 

(i) a pixel locatx>r and segmentor; 

(ii) a $h£^e analyser; 

(ili) a feising anal^er; and 
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(iv) a pixel modifier. 

40. A method according to cfmm 39 wherein ssM ptxet {ocator and se^entxir includes a 
pixel trsjvsfOftmeF, 

5 

41. A method as claimed in claim 26 comprising «ie step of: responsive to determining that 
corrective processing is necessary but cannot be beneficially applied to said first acquired image 
according to smd characteristics, disabling said step of detecting red-eye defects md providing an 
Indication of such to a user. 

i 0 42- A mefliod as claimed in claim 3 1 vwfeerein said stsp of applying a oottective proems 
includes interacting with a user to determine the coirections to be made to said third image. 

43. A method acconling to claim 26 wherein the step of ^plying a conective process 
inclades p«fbrming a color space transfotmation. 

15 44. A method as claimed in claim 25 in which one or more of st^ a) to e) are performed in a 
digital camera. 

45. A method as claimed in claim 26 comprising determining a sequence in which more thm 
one corrective process can be beneliciatly applied to said ilrst acquired image according to said 

20 characteristics. 

46. A computer-readabie ;^orage medium containing a set of instructions which when 
executed on a digitel image jwocessiag device perform the steps of claim 25. 

25 47. A digital image processing device arranged to perform the steps of claim 25. 
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